Cache management in content delivery systems

ABSTRACT

Examples described herein relate to apparatuses and methods for managing caching for a content delivery system, which may include receiving a content request indicating that the caching agent is requesting content data for a client, filling the content data in a first cache storage of the business logic agent, providing the cached content data to the caching agent, and while a second cache storage of the caching agent is being filled with the content data, maintaining the cached content data in response to receiving additional content requests from the caching agent. The additional content requests may indicate that the caching agent is requesting the same content data for additional clients.

BACKGROUND

A content delivery system or network (e.g., a content delivery network(CDN)) is a geographically distributed network of servers configured forfacilitating an origin server to distribute content data (e.g., videos,images, website content data, and so on) of the origin server to clientsthat consume the content data. Each server in the content deliverysystem can be referred to as a node, a machine, and so on. To distributethe content data from the origin server to clients that aregeographically remote to the origin server, a node in geographicalproximity to the clients can provide the content data to those clientson behalf of the origin server. In particular, the CDN can replicate andcache the content data of the origin server and provide the replicatedand cached content data to the clients.

BRIEF SUMMARY

Provided herein are systems, apparatuses, and methods for managingcaching for a content delivery system, which may include receiving, by abusiness logic agent from a caching agent, a content request indicatingthat the caching agent is requesting content data for a client, whereinthe business logic agent and the caching agent are operatively coupledvia a network, filling, by the business logic agent, the content data ina first cache storage of the business logic agent, wherein the contentdata is received from an upstream node of the content delivery system,providing, by the business logic agent, the cached content data to thecaching agent, and while a second cache storage of the caching agent isbeing filled with the content data, maintaining, by the business logicagent, the cached content data in response to receiving additionalcontent requests from the caching agent, wherein the additional contentrequests indicate that the caching agent is requesting the same contentdata for additional clients.

The content delivery system may be a CDN. The business logic agent andthe caching agent may be on the same edge node of the CDN. The upstreamnode is preferably an origin server that stores the content data or anode between the edge node and the origin node in the CDN that storesthe content data. The network may be a local host connection. The localhost connection may be over at least one of a hypertext transferprotocol (HTTP) connection and a HTTP/2 connection.

Alternatively, the caching agent is on an edge node of the CDN, and thebusiness logic agent is on an intermediate node of the CDN. Theintermediate node is preferably different from the edge node.

The caching agent may include a HTTP service engine configured toservice HTTP requests received from clients. The caching agent mayinclude a caching engine. The caching engine may include the secondcache storage.

Embodiments further allow receiving, by the business logic agent fromthe caching agent, an authentication request before the content requestis received, authenticating that the content data is provided by acustomer of the content delivery system, and sending, by the businesslogic agent to the caching agent, an authentication response in responseto authenticating that the content data is provided by the customer,wherein the authentication response comprises an authorization.

The content request may identify the upstream node. The business logicagent may receive the content data from the upstream node based on thecontent request. The content request may be addressed to the upstreamnode.

The content request may include an address of the content data and arange of the content data. The address of the content data may include aUniform Resource Locator (URL). The business logic agent can determinewhether any of the additional content requests is requesting the samecontent data as requested by the content request based on the addressand the range.

The content request may include a fill identification (ID), and themethod further may include determining, by the business logic agent,that a new copy of the content data is needed based on the fill ID, andfilling, by the business logic agent, the new copy of the content datain the first cache storage in response to determining that the new copyis needed.

The content request may include a conditional request header, and themethod further allows determining, by the business logic agent, a keycorresponding to the content request based on the conditional requestheader.

Embodiments further allow, while the second cache storage of the cachingagent is being filled with the content data, maintaining the cachedcontent data to include maintaining the cached content data in responseto determining that two consecutive ones of the additional contentrequests are received within a predetermined time interval.

Embodiments further allow determining that no new additional contentrequests have been received for a predetermined time interval since amost recent one of the additional content requests has been received.

In accordance with another aspect, systems, apparatuses, and methods formanaging caching for a content delivery system may include receiving, bya caching agent from a client, a request for content data, determining,by the caching agent, that the caching agent needs to fill the contentdata, sending, by the caching agent to a business logic agent, a contentrequest indicating that the caching agent is requesting the content datafor the client, wherein the business logic agent and the caching agentare operatively coupled via a network, and the business logic agentfills the content data in a first cache storage of the business logicagent responsive to the content request, filling, by the caching agent,the content data from an upstream node in a second cache storage of thecaching agent, receiving, by the caching agent, additional contentrequests from additional clients, wherein the additional contentrequests indicate that the same content data is requested by additionalclients, and while the second cache storage is being filled with thecontent data, the cached content data is provided by the business logicagent to the additional clients.

Embodiments further allow selecting, by the caching agent, the upstreamnode, wherein the content request identifies the upstream node.

These and other features, together with the organization and manner ofoperation thereof, will become apparent from the following detaileddescription when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram of a content delivery system according to someembodiments.

FIG. 1B is a diagram of a content delivery system according to someembodiments.

FIG. 2 is a block diagram that illustrates a node according to someembodiments of the present disclosure.

FIG. 3A is a flow diagram illustrating a method for managing caching forthe content delivery system according to various embodiments.

FIG. 3B is a flow diagram illustrating a method for managing caching forthe content delivery system according to various embodiments.

DETAILED DESCRIPTION

Embodiments described herein relate to systems, apparatuses, and methodsfor cache management in a content delivery system (e.g., a CDN). In acontent delivery system, an edge node is a node that initially receivesa request for content data from a client. The client refers to a deviceoperated by an end user who desires to consume or otherwise receive thecontent data provided by the origin server. The content data refers to aportion, a segment, an object, a file, or a slice of data stored by theorigin server and cached throughout the content delivery system forprovisioning to the clients. The origin server refers to a deviceoperated by a customer of the content delivery system, which facilitatesthe customer in delivering the content data to the clients. Nodes of acontent delivery system can be organized in a caching hierarchy forcaching content data requested by clients.

In some implementations, the edge node has two separate agents providedthereon for caching and business logic functionalities. For example, acaching agent is configured to process the requests for content (e.g.,hypertext transfer protocol (HTTP) requests) received from clients, andreceive and cache the requested content data in case additional clientsrequest the same content data, for example, if the content data (e.g.,at least a portion of a video) is popular and is frequently requested. Abusiness logic agent is configured to implement business logic for theedge node. For example, the business logic agent can determine whetherthe content data requested belongs to a customer of the content deliverysystem, whether the content delivery system can deliver the content datato the client requesting the content data (assuming the content databelongs to a customer of the content delivery system), and so on.Preferably, the caching agent and the business logic agent havedifferent functionalities and are implemented as separate agents. Inother words, the business logic agent is external to the caching agent,and vice versa.

In accordance with some aspects, upon receiving a request for contentdata from a client, the caching agent obtains authorization from thebusiness logic agent, allowing the request to be serviced. Next, thecaching agent determines whether the requested content data is alreadystored in a cache local to the caching agent. Responsive to determiningthat the cache local to the caching agent does not store the contentdata, the caching agent determines that the cache local to the cachingagent needs to be filled with the content data to serve future requestsreceived from additional clients. In order to satisfy the request, thecaching agent fetches the content data from the origin server or anotherupstream node that stores or caches the content data and performs acache fill (e.g., filling the cache local to the caching agent with thecontent data). The caching agent can provide to the client a copy of thecontent data stored in the cache local to the caching agent.

In a low latency environment, as the caching agent performs a cachefill, the caching agent may enable a cache lock. With the cache lock on,any additional requests (from additional clients) for the same contentdata are placed on hold until the cache fill is completed. This meansthat while the first client that first requests the content data (beforethe additional clients) may obtain the content data immediately (e.g.,as the cache fill is in progress), the additional clients that requestthe same content data will not obtain any of the content data until thecache fill is completed, until the cache lock is disabled, and until thefirst client obtains all the content data.

Traditionally, the caching agent does not have any fill-in-progressfeatures that allows servicing of the additional requests before thecache fill at the caching agent is completed (before the cache lock isdisabled). In some traditional implementations of the caching agent, thecaching agent does not have sufficient signaling mechanism toefficiently implement any fill-in-progress features. Some traditionalimplementations of the caching agent allow the additional clients topoll to determine whether the cache lock has been released. The pollinterval (e.g., 500 ms) may be set to avoid frequent polling whichwastes central processing unit (CPU) cycles. In that regard, time iswasted if the cache lock is disabled long before a given poll intervalexpires. The time needed to perform the cache fill may also vary basedon a size of the content data (e.g., resource size) such that pollingcannot be implemented efficiently.

Moreover, some traditional implementations of the caching agent directthe additional clients to connect to the origin server or anotherupstream node that already has the content data stored/cached while thecache lock is enabled so that the additional clients can obtain thecontent data directly from the original server or the upstream node.Given that the content delivery system is a geographically distributednetwork, the connections between the additional clients and the originserver or another upstream node over the distributed network may requirea significant amount of bandwidth, may exacerbate traffic within thecontent delivery system, may swamp the customer's origin server, and soon.

Embodiments described herein improve cache management in traditionalcontent delivery systems that implement a caching agent (for processingrequests from clients and for caching content data) and a business logicagent (for implementing business logics such as authentication). Forexample, while the caching agent has a cache lock enabled as a cachelocal to the caching agent is being filled with requested content data,the business logic agent maintains a copy of the content data in a cachelocal to the business logic agent. Additional requests for the samecontent data received by the caching agent can be serviced using thecopy of the content data stored in the cache local to the business logicagent while the cache local to the caching agent is unavailable toservice those requests due to the cache lock. Accordingly, the cachelocal to the business logic agent may act as an intelligentfill-buffering proxy for the caching agent during the cache lock.

FIG. 1A is a diagram of a content delivery system 100 a according tosome embodiments. Referring to FIG. 1A, the content delivery system 100a is configured for delivering content data provided by an origin server150 to various clients 102 a-102 n. As shown, each of the users 101a-101 n operates or is associated with a respective one of the clients102 a-102 n for accessing the content data or services provided by theorigin server 150. In some embodiments, each of the clients 102 a-102 ncan be a desktop computer, mainframe computer, laptop computer, paddevice, smart phone device, or the like, configured with hardware andsoftware to perform operations described herein. For example, each ofthe clients 102 a-102 n includes at least a processing circuit, anetwork device, and a user interface. The processing circuit isconfigured to perform functions of the clients 102 a-102 n describedherein. The network device is configured to connect the clients 102a-102 n to a node (e.g., an edge node 110) of the content deliverysystem 100 a. The user interface is configured for outputting (e.g.,displaying media content, games, information, and so on) based on thecontent data as well as receiving user input from the users 101 a-101 n.

In some examples, the content delivery system 100 a corresponds to a CDNfor delivering and distributing the content data originating from theorigin server 150 to the clients 102 a-102 n. For example, the contentdelivery system 100 a includes nodes 110, 140, . . . , and 150, wherethe origin server 150 is connected to at least one node (not shown), oneof the at least one node is connected to the node 140, and the node 140is connected to the edge node 110. The origin server 150, the node 140,the edge node 110, and other nodes in the content delivery system 100 anot shown can be located in different locations, thus forming thegeographically distributed content delivery system 100 a. While therecan be additional nodes between the node 140 and the origin server 150,the node 140 can be directly connected to the origin server 150, or thenode 140 can be the origin server 150.

The content data of the origin server 150 can be replicated and cachedin multiple locations (e.g., multiple nodes) throughout the contentdelivery system 100 a, including in the node 140 and other nodes (notshown). As used herein, the node 140 refers to any node in the contentdelivery system 100 a (between the origin server 150 and the edge node110) that stores a copy of the content data provided by the originserver 150. The origin server 150 refers to the source of the contentsdata. The origin server 150 can belong to a customer (e.g., a contentowner, content publisher, or a subscriber of the system 100 a) of thecontent delivery system 100 a such that the customer pays a fee forusing the content delivery system 100 a to deliver the content data.Examples of the content data include but are not limited to, webpagesand web objects (e.g., text, graphics, scripts, and the like),downloadable objects (e.g., media files, software, documents, and thelike), live streaming media, on-demand streaming media, social networks,and applications (e.g., online multiplayer games, dating applications,e-commerce applications, portals, and the like), and so on.

The nodes 110, 140, and other nodes (not shown) between the edge node110 and the origin server 150 form a “backbone” of the content deliverysystem 100 a, providing a path from the origin server 150 to the clients102 a-102 n. The node 140 is upstream with respect to the edge node 110given that the node 140 is between the edge node 110 and the originserver 150. The nodes making up a backbone may be dynamically orstatically selected based on the location of those nodes, taking intoconsideration a number hops or links from the origin server 150 to theclients 102 a-102 n, latency, availability, cost, and other suitablecriteria.

In some embodiments, the edge node 110 is referred to as an “edge node”given the proximity of the edge node 110 to the clients 102 a-102 n. Forexample, the clients 102 a-102 n that are in an area 105 may beassociated with and connected to the edge node 110 given the proximityof the edge node 110 to the clients 102 a-102 n. In other words, theedge node 110 is on the edge of the content delivery system 100 a, andthe edge node 110 is directly connected to the clients 102 a-102 n.Typically, the closer an edge node is to clients connected thereto, theless latency those clients experience with respect to receiving thecontent data from that edge node. Thus, performance is contingent uponthe geographical proximity of the edge node 110 to the clients 102 a-102n. CDN providers typically place the edge nodes as close to intendedclients as possible. Thus, the edge node 110 can be located within thearea 105. In some embodiments, the edge node 110 may be directlyconnected to the origin server 150.

In some embodiments, the node 140 (and other nodes between the node 140and the origin server 150 not shown) is referred to as an “intermediatenode.” The intermediate nodes link the edge nodes to the origin server150 via various network links or “hops.” The intermediate nodes canprovide the content data (and updates thereof) to the edge nodes. Thatis, the origin server 150 can provide the content data (and updatesthereof) to the edge node 110 through the node 140, if the edge node 110does not currently cache a copy of the content data requested by theclients 102 a-102 n.

Each link between one of the clients 102 a-102 n and the edge node 110corresponds to a suitable network connection for exchanging data. Inaddition, each link between two of the nodes/servers 110, 140, . . . ,and 150 represents a suitable network connection for exchanging data. Anetwork connection is structured to permit the exchange of data, values,instructions, messages, and the like among the clients 102 a-102 n, thenodes 110, 140, and so on, and the origin server 150 in the mannershown. The network connection can be any suitable Local Area Network(LAN) or Wide Area Network (WAN) connection. For example, each networklink can be supported by Frequency Division Multiple Access (FDMA), TimeDivision Multiple Access (TDMA), Synchronous Optical Network (SONET),Dense Wavelength Division Multiplexing (DWDM), Optical Transport Network(OTN), Code Division Multiple Access (CDMA) (particularly,Evolution-Data Optimized (EVDO)), Universal Mobile TelecommunicationsSystems (UMTS) (particularly, Time Division Synchronous CDMA (TD-SCDMAor TDS) Wideband Code Division Multiple Access (WCDMA), Long TermEvolution (LTE), evolved Multimedia Broadcast Multicast Services(eMBMS), High-Speed Downlink Packet Access (HSDPA), and the like),Universal Terrestrial Radio Access (UTRA), Global System for MobileCommunications (GSM), Code Division Multiple Access 1x RadioTransmission Technology (1x), General Packet Radio Service (GPRS),Personal Communications Service (PCS), 802.11X, ZigBee, Bluetooth,Wi-Fi, any suitable wired network, combination thereof, and/or the like.

As shown, the edge node 110 includes a caching agent 120 and a businesslogic agent 130. The caching agent 120 can be an open-source platformand is configured to process requests for content (e.g., HTTP requests)received from the clients 102 a-102 n, and receive and cache therequested content data in case additional clients request the samecontent data. In that regard, the caching agent 120 includes a requestservice engine 122 and a caching engine 124.

An example of the request service engine 122 is a HTTP service engineconfigured to receive and process HTTP requests received from theclients 102 a-102 n. The request service engine 122 is configured forsuitable protocols (e.g., HTTP) for receiving and processing the HTTPrequest. In other words, the request service engine 122 is configured toanswer the HTTP requests from the end users 101 a-101 n in the mannerdescribed.

The caching engine 124 is configured to implement caching by the cachingagent 120. For example, the caching engine 124 includes or isoperatively coupled to a cache storage 126. The cache storage 126 islocal to the caching engine 124 and to the caching agent 120. Contentdata cached by the caching agent 120 (e.g., by the caching engine 124)is stored in the cache storage 126.

The business logic agent 130 is configured to implement business logicsat the edge node 110. For example, the business logic agent 130 includesa business logic engine 132. The business logic engine 132 is configuredfor authentication, providing business information to the caching agent120 to allow the caching agent 120 to maintain correct statistics andlogs, calculating cache key, and so on. For example, the business logicengine 132 is configured to determine whether the content data requestedby the clients 102 a-102 n belongs to a valid customer of the contentdelivery system 100 a, whether the rules of the customer allow thecontent data to be serviced to the clients 102 a-102 n, whether therules of the content delivery system 100 a allow the content data to beserviced to the clients 102 a-102 n, and so on.

Furthermore, the business logic agent 130 includes a fill-buffer proxyengine 134 configured to implement an intelligent fill-buffering proxyfor the caching agent 120 when the content data stored in the cachestorage 126 is cache locked. For example, the fill-buffer proxy engine134 includes or is operatively coupled to a cache storage 136. The cachestorage 136 is local to the fill-buffer proxy engine 134 and to thebusiness logic agent 130. Content data cached by the business logicagent 130 (e.g., by the fill-buffer proxy engine 134) is stored in thecache storage 136. The business logic agent 130 including both thebusiness logic engine 132 and the fill-buffer proxy engine 134 indicatesthat both authentication and fill-buffering services can be implementedon a same process, by virtue that a same connection opened by thebusiness logic agent 130 is used for both authentication andfill-buffering by both the business logic engine 132 and the fill-bufferproxy engine 134. Alternatively, the business logic engine 132 and thefill-buffer proxy engine 134 can be executed on separate processes on anode (e.g., the edge node 110 or the node 111).

As shown in FIG. 1A, the caching agent 120 and the business logic agent130 are provided on a same node (the edge node 110), same server, samecomputer, same “box,” and so on. The caching agent 120 and the businesslogic agent 130 have different functionalities and are implemented asseparate agents. The caching agent 120 and the business logic agent 130are implemented by the processing and storage capabilities of the sameedge node 110. In other words, the business logic agent 130 is externalto the caching agent 120 on the same edge node 110, vice versa.Providing the two agents 120 and 130 on the same edge node 110 allowseparation of the request servicing/caching logic and the businesslogic, thus providing granularity for managing the separate logics andfor providing updates to the software. In some examples in which thecaching agent 120 already has upstream target selection capabilities,there is no need to replicate upstream target selection at the businesslogic agent 130. If it has been detected that the business logic agent130 is faulty, the fill-buffer capabilities as described herein can bereleased or removed from the business logic agent 130, and the cachelock features can address multiple requests for a same content data withcache lock enabled (although less efficient).

As shown in FIG. 1A, the caching agent 120 and the business logic agent130 are operatively coupled to one another via a network 140 a. In someexamples, the network 140 a is a local host connection configured toprovide communications for two agents on a same node. In some examples,the local host connection is a HTTP connection, a HTTP/2 connection, atransmission control protocol (TCP) connection, and so on. In someexamples, the local host connection is peripheral component interconnect(PCI) connection in the embodiments in which the caching agent 120 andthe business logic agent 130 are implemented on different hardware(e.g., different processors, different memories, and so on) of the edgenode 110. Therefore, the network 140 a is an intra-node networkconnection.

As described in further detail herein, the present disclosure improvesthe dual-agent scheme by leveraging the connection via the network 140 abetween the caching agent 120 and the business logic agent 130 for bothauthentication as well as fill-buffering. A separate connection isopened for each request received by the request service engine 122 fromone of the clients 102 a-102 n. Multiple requests (e.g., theauthentication request, the content request, and so on) forauthentication and fill-buffering proxy with respect to a same requestreceived by the request service engine 122 from one of the clients 102a-102 n can be communicated and serviced serially on the sameconnection. In other words, this is connection is “kept-alive.” In someexamples, multiplexed servicing of multiple authentication requests andcontent requests can be allowed on the same connection, e.g., as wouldbe the case with the HTTP/2 implementation.

FIG. 1B is a diagram of a content delivery system 100 b according tosome embodiments. Referring to FIGS. 1A and 1B, the content deliverysystem 100 b is configured for delivering content data provided by theorigin server 150 to various clients 102 a-102 n. The content deliverysystem 100 b is similar to the content delivery system 100 a, butdiffers from the content delivery system 100 a in that in the contentdelivery system 100 b, the caching agent 120 and the business logicagent 130 are provided on different nodes 110 and 111. For example, thecaching agent 120 can be provided on the edge node 110 for requestservicing and caching functionalities, and the business logic agent 130is provided on another node 111 for business logic functionalities. Thenodes 111 and 140 are upstream nodes to the edge node 110.

The caching agent 120 and the business logic agent 130 are operativelycoupled via a network 140 b. The network 140 b is structured to permitthe exchange of data, values, instructions, messages, and the likebetween the caching agent 120 and the business logic agent 130. Thenetwork 140 b can be any suitable LAN or WAN connection. For example,the network 140 b can be supported by FDMA, TDMA, SONET, DWDM, OTN, CDMA(particularly, EVDO), UMTS (particularly, TD-SCDMA or TDS WCDMA, LTE,eMBMS, HSDPA, and the like), UTRA, GSM, 1x, GPRS, PCS, 802.11X, ZigBee,Bluetooth, Wi-Fi, any suitable wired network, combination thereof,and/or the like. Therefore, the network 140 b is an inter-node networkconnection.

While one node (the edge node 110) is shown to be connected to the node111, one or more additional edge nodes (such as but not limited to, theedge node 110) can be connected to the node 111. In other words,multiple edge nodes configured for request servicing and caching can beconnected to a same intermediate node configured for business logic.Such node deployment allow efficient allocation of hardware. Forexample, the edge node 110 with the caching agent 120 may have largerstorage capability and lesser compute capability given that the edgenode 110 is concerned with caching. In some examples, basic compute canbe provided to the edge node 110 for HTTP processing. The node 111 withthe business logic agent 130 may have larger compute capability andlesser storage capability given that the node 111 is concerned with thebusiness logic (e.g., authentication, rule checks, and so on).

In some examples, the fill-buffering proxy engine 134 can be separatefrom the business logic engine 132. For example, the fill-bufferingproxy engine 134 and the business logic engine 132 can be separateprocesses/applications operating on a same node. Alternatively, thefill-buffering proxy engine 134 and the business logic engine 132 can beseparate processes/applications operating on different nodes andconnected via a suitable network. In such embodiments, an indication canbe included in the authentication request indicating whether onlyauthentication/metadata is requested (e.g., the caching engine 124already has a copy of the content data) or whether (assuming successfulauthentication) the content data should be retrieved as well.

FIG. 2 is a block diagram that illustrates a node 200 according to someembodiments. Referring to FIGS. 1A-2 , the node 200 is a non-limitingexample of the nodes 110, 111, 140, and nodes (if any) between the node140 and the origin server 150 in some embodiments. As shown, the node200 includes one or more of a processing circuit 210 and a networkdevice 220.

The processing circuit 210 is configured to perform various functionsdescribed herein relative to the node 200. The processing circuit 210includes a processor 212 and a memory 214. The processor 212 can beimplemented with a general-purpose processor, an Application SpecificIntegrated Circuit (ASIC), one or more Field Programmable Gate Arrays(FPGAs), a Digital Signal Processor (DSP), a group of processingcomponents, or other suitable electronic processing components. Thememory 214 can be implemented with a Random Access Memory (RAM),Read-Only Memory (ROM), Non-Volatile RAM (NVRAM), flash memory, harddisk storage, or another suitable data storage unit. The memory 214stores data and/or computer code for facilitating the various processesexecuted by the processor 212. Moreover, the memory 214 is or includestangible, non-transient volatile memory or non-volatile memory.Accordingly, the memory 214 includes database components, object codecomponents, script components, or any other type of informationstructure for supporting the various functions described herein.

In some examples, the processing circuit 210 of the edge node 110 (FIG.1A) is configured to implement the caching agent 120 and the businesslogic agent 130 as separate agents on the edge node 110. The memory 214of the edge node 110 (FIG. 1A) can be used to implement the cachestorages 126 and 136 as separate storages operatively coupled toseparate agents. In one example, the cache storages 126 and 136correspond to separate petitions on the memory 214. In another example,the memory 214 may include separate memory devices, each connected toother components of the edge node 110 (FIG. 1A) via a suitable internalbus. The cache storage 126 is implemented on a first one of the separatememory devices, and the cache storage 136 is implemented on a second oneof the separate memory devices.

In some examples, the processing circuit 210 of the edge node 110 (FIG.1B) is configured to implement the caching agent 120, and the processingcircuit 210 of the node 111 (FIG. 1B) is configured to implement thebusiness logic agent 130. The memory 214 of the edge node 110 (FIG. 1B)can be used to implement the cache storage 126, and memory 214 of thenode 111 (FIG. 1B) can be used to implement the cache storage 136.

The network interface 220 is structured to establish communication withclients (e.g., the clients 102 a-102 n), other nodes in the contentdelivery system 100 a or 100 b, and/or the origin server 150. In someexamples, the network interface 220 is configured to establish thenetwork 140 a or 140 b. The network interface 220 includes hardware andsoftware for achieving such. In some implementations, the networkinterface 220 includes a cellular transceiver (configured for cellularstandards), a local wireless network transceiver (for 802.11X, ZigBee,Bluetooth, Wi-Fi, or the like), a wired network interface, a combinationthereof (e.g., both a cellular transceiver and a Bluetooth transceiver),and/or the like.

FIG. 3A is a flow diagram illustrating a method 300 a for managingcaching for the content delivery system 100 a or 100 b according tovarious embodiments. Referring to FIGS. 1A-3A, the method 300 a isconcerned with leveraging the cache storage 136 of the business logicagent 130 for intelligent fill-buffering for the caching agent 120 withrespect to content data for which the caching agent 120 has enabled acache lock. The method 300 a is performed by the caching agent 120.

At 302, the caching agent 120 (e.g., the request service engine 122)receives a request for content data from the client 102 a. An example ofthe request is an HTTP request. The request can be received from thecloud and/or from the internet. The client 102 a refers to a first oneof the clients 102 a-102 n that is requesting the content data from theedge node 110 for the method 300 a.

In some examples, responsive to receiving the request, the caching agent120 (e.g., the request service engine 122) opens a connectioncorresponding to that particular request in the network 140 a or 140 b.For example, the request service engine 122 can open a portcorresponding to the request received from the client 102 a in the localhost network (e.g., the network 140 a) for communicating therequest-related data with the business logic agent 130. In anotherexample, the request service engine 122 can open a dedicated connectioncorresponding to the request received from the client 102 a in thenetwork 140 b for communicating request-related data with the businesslogic agent 130.

The caching agent 120 (e.g., the request service engine 122) sends anauthentication request to the business logic engine 132 via the network140 a (e.g., via the opened port) or 140 b (e.g., via the dedicatedconnection) to authenticate the request received from the client 102 a.The business logic engine 132 is configured to determine whether thecontent data requested by the client 102 a belongs to a valid customerof the system 100 a or 100 b, whether the rules of the customer allowthe content data to be serviced to the client 102 a, whether the rulesof the content delivery system 100 a or 100 b allow the content data tobe serviced to the client 102 a, and so on. The request service engine122 receives an authentication response from the business logic engine132. The authentication response indicates to the request service engine122 whether the request received from the client 102 a can be serviced(e.g., “yes” or “no”). In some examples, in addition to the indicationof whether the request can be serviced, the authentication response caninclude metadata such as but not limited to, server identification (ID),subscriber ID, cache key to use, and so on. In response to determiningthat the request received from the client 102 a cannot be serviced basedon the authentication response, the request service engine 122 respondto the client 102 a accordingly.

On the other hand, in response to determining that the request receivedfrom the client 102 a can be serviced based on the authenticationresponse, the caching engine 124 determines whether the caching agent120 needs to fill the content data. For example, at 304, the cachingengine 124 determines whether the requested content data is cached inthe cache storage 126. Responsive to determining that the content datais cached in the cache storage 126 (304: YES), the caching agent 120provides the content data to the client 102 a from the cache storage 126of the caching agent 120, at 306.

On the other hand, in response to determining that the content data isnot cached in the cache storage 126 (304: NO), the caching agent 120(e.g., the caching engine 124) sends a content request to thefill-buffer proxy engine 134 indicating that the caching agent 120 isrequesting the content data for the client 102 a, at 310. The contentrequest can be sent via the port or connection established in thenetwork 140 a or 140 b (as dedicated to the request from the client 102a) for authentication.

In some examples, the content request includes an address of the contentdata and an address of an upstream node from which the content data isto be received/retrieved. In some examples, the address of the contentdata is a Uniform Resource Locator (URL). The upstream node is referringto a node that stores a copy of the content data requested by the host102 a. Examples of the upstream node includes the node 140 and theorigin server 150. The caching engine 124 can select the upstream node.The caching engine 124 has suitable caching tools including but notlimited to, upstream node selection tools (e.g., through consistenthashing), health status information to track health (e.g., latency,capacity, and so on) of the nodes of the systems 100 a or 100 b, and soon. The caching engine 124 can use such caching tools to identify ahealthy upstream node that currently stores a copy of the content datafor example, based on availability of the copy of the content data,latency, proximity to edge node 110 (e.g., in the system 100 a and 100b), proximity to the node 111 (e.g., in the system 100 b), latency,availability/health, and so on. The address of the upstream node can bea node/server ID, an IP address, and so on.

In some examples, the fill-buffer proxy engine 134 appears to be“transparent” to the caching agent 120. For example, a protocol can beimplemented at the caching agent 120 (e.g., the caching engine 124) suchthat although the caching agent 120 configures the content request to besent to the upstream node (according to the IP address of the upstreamnode), the protocol configures the caching agent 120 to connect insteadto the fill-buffer proxy engine 134 using a known IP address of thefill-buffer proxy engine 134. In other words, the protocol converts thereceiver address from the address of the upstream node to the knownaddress of the fill-buffer proxy engine 134 while maintaining theaddress of the upstream node in the content request to the fill-bufferproxy engine 134. The fill-buffer proxy engine 134 does not perform nodeselection given that the caching agent 120 already has the caching toolsavailable.

At 312, the caching engine 124 begins filling the content data in thecache storage 126 and enables cache lock in the cache storage 126 withrespect to the content data. In some embodiments, responsive toreceiving the content request, the fill-buffer proxy engine 134 connectsto the upstream node identified in the content request and receives thecontent data from the upstream node. The fill-buffer proxy engine 134begins to fill the content data in the cache storage 136 and relays thecontent data to the caching agent 120 via the port or connection of thenetwork 140 a or 140 b. Accordingly, the cache storage 126 is beingfilled with the content data from the upstream node via the fill-bufferproxy engine 134, which is acting as a proxy or relay between theupstream node and the edge node 110. While the cache storage 126 isbeing filled with the content data, the cache lock with respect to thecontent data is enabled. As the cache storage 126 is being filled withthe content data, the content data can be sent to the client 102 a suchthat the client 102 a does not need to wait until the entirety of thecontent data to be filled in the cache storage 126 before getting thecontent data. However, any additional requests for the same content datafrom another client cannot be serviced from the cache storage 126 aslong as the cache lock is enabled.

At 314, the caching agent 120 receives an additional request for thesame content data from an additional client 102 b. Based on the request(e.g., an HTTP request), the caching agent 120 can determine that theadditional request is for the same content but from a different client.

In some examples, responsive to receiving the additional request, thecaching agent 120 (e.g., the request service engine 122) opens aconnection corresponding to that particular request in the network 140 aor 140 b. For example, the request service engine 122 can open a portcorresponding to the request received from the client 102 b in the localhost network (e.g., the network 140 a) for communicating data with thebusiness logic agent 130. The port is different from the port openedwith respect to the request received at 302. In another example, therequest service engine 122 can open a dedicated connection correspondingto the request received from the client 102 b in the network 140 b forcommunicating data with the business logic agent 130. The dedicatedconnection is different from the dedicated connection opened withrespect to the request received at 302.

In some embodiments, the caching agent 120 (e.g., the request serviceengine 122) sends an authentication request to the business logic engine132 via the network 140 a or 140 b to authenticate the request receivedfrom the client 102 b, in a manner similar to described with respect tothe request received at 302. The request service engine 122 receives anauthentication response from the business logic engine 132, where theauthentication response indicates to the request service engine 122whether the request received from the client 102 b can be serviced. Asdescribed, the authentication response can additionally include metadatasuch as but not limited to, server ID, subscriber ID, cache key to use,and so on. In response to determining that the request received from theclient 102 b cannot be serviced based on the authentication response,the request service engine 122 respond to the client 102 b accordingly.

On the other hand, in response to determining that the request receivedfrom the client 102 b can be serviced based on the authenticationresponse, the caching engine 124 determines whether filling the contentdata at the cache storage 126 has completed (e.g., whether the cachelock has been disabled or released), at 316. Responsive to determiningthat the fill is completed and/or that the cache lock has been disabledor released (316: YES), the caching engine 124 provides the content datato the client using the cache storage 126, at 318.

On the other hand, responsive to determining that the fill is notcompleted and/or that the cache lock has not been disabled or released(316: NO), the cached content data is provided by the fill-buffer proxyengine 134 to the additional client 102 b. The cached content data isprovided by the fill-buffer proxy engine 134 to the client 102 b whilethe cache storage 126 is being filled with the content data. Forexample, responsive to determining that the fill is not completed and/orthat the cache lock has not been disabled or released (316: NO), thecaching engine 124 is configured to send an additional content requestto the fill-buffer proxy engine 134, at 320. The additional contentrequest can be sent via the port or connection established in thenetwork 140 a or 140 b (as dedicated to the client 102 b) forauthentication. The caching engine 124 can select the upstream node forthis additional request in the manner described. Similar to the contentrequest sent at 310, the additional content request includes an addressof the content data and an address of an upstream node from which thecontent data is to be received/retrieved. As described, with respect tothe additional content request, the fill-buffer proxy engine 134 appearsto be “transparent” to the caching agent 120 in the manner described.

In some implementations, the systems 100 a and 100 b can employ largeobject slicing. Given that resources can be several gigabytes or more insize, the resources can be broken down or sliced into multiple portionsor slices to reduce efficiency, latency, and bandwidth issues. The sizesof the slices can be standardized to be the same. The slices that makeup a given resource can correspond to an HTTP range. In other words, theHTTP range further identifies a particular slice of the content data,given that the address of the content data (e.g., the URL) does notspecific particular slices of the content data. Accordingly, the contentrequest can further include a range of the content data in addition tothe address of the content data. The range can be specified in a rangeheader on the content request. The fill-buffer proxy engine 134 uses therange header as part of a key for deciding whether the request receivedat 314 corresponds to the same content data. In other words, thefill-buffer proxy engine 134 can determine whether any of the additionalcontent requests is requesting the same content data based on theaddress and the HTTP range. In one example in which the fill-bufferproxy engine 134 is already filling or storing a full version of thecontent data (identified by the address of the content data) at thecache storage 136 where the content data has multiple slices/ranges, thecontent request received at 320 includes a range header that identifiesa particular slice of the content data. The fill-buffer proxy engine 134can identify that particular slice based on the range header and servicethe slice of the content data in the manner described from the fullversion stored in the cache storage 136.

At 322, the content data is provided to the additional client 102 b viathe fill-buffer proxy engine 134. In the example in which the additionalcontent request identifies the same upstream node, the fill-buffer proxyengine 134 has already established a connection with the upstream nodefor processing the first content request with respect to the client 102a. The same connection between the fill-buffer proxy engine 134 and theupstream node can be leveraged to service the same content data to theadditional request from the client 102 b. In the example in which thecontent request identifies a different upstream node, the fill-bufferproxy engine 134 is configured to establish a connection with thedifferent upstream node for receiving the content data.

In some embodiments, the block 322 includes timing out the requestreceived at 314 and direct the client 102 b to connect to thefill-buffer proxy engine 134 instead to receive the content data fromthe cache storage 136 while the cache lock is enabled at the cachestorage 126. For example, responsive to determining that the fill is notcompleted and/or that the cache lock has not been disabled or released(316: NO), the request service engine 122 times out the request receivedat 314 (e.g., setting timeout=0) and sends the client 102 b a messagewith an address (e.g., the IP address) of the fill-buffer proxy engine134, indicating to the client 102 b to connect to the fill-buffer proxyengine 134 for receiving the content data. The client 102 b can connectto the fill-buffer proxy engine 134 and receives the content data storedin the cache storage 136.

In alternative embodiments, the block 322 includes the caching agent 120receiving the content data via the port or connection established in thenetwork 140 a or 140 b (as dedicated to the client 102 b) forauthentication and relaying the content data to the client 102 b. Insome examples, a dedicated tunnel can be established from thefill-buffer proxy engine 134 to the client 102 b via the port orconnection established for authentication and through the caching agent120. The content data can be transferred from the fill-buffer proxyengine 134 to the client 102 b via the dedicated tunnel.

The method 300 a returns to block 314 as additional requests fromadditional ones of the clients 102 a-102 n are received by the cachingagent 120 in the manner described. As described in further detailsherein, as long as there are requests for a given content beingreceived, the fill-buffer proxy engine 134 maintains a copy of thecontent data in the cache storage 136 and maintains the upstreamconnection to the upstream node. Responsive to determining that noadditional requests for the same content data are being received, thecontent data can be cleared from the cache storage 136, and theconnection of the fill-buffer proxy engine 134 to the upstream node canbe terminated.

FIG. 3B is a flow diagram illustrating a method 300 b for managingcaching for the content delivery system 100 a or 100 b according tovarious embodiments. Referring to FIGS. 1A-3B, the method 300 b isconcerned with leveraging the cache storage 136 of the fill-buffer proxyengine 134 for intelligent fill-buffering for the caching agent 120 withrespect to content data for which the caching agent 120 has enabled acache lock. The method 300 b is performed by the business logic agent130 and corresponds to the method 300 a.

As described, the business logic agent 130 (e.g., the business logicengine 132) can receive an authentication request from the caching agent120 (e.g., the request service engine 122) via the network 140 a (e.g.,via the opened port) or 140 b (e.g., via the dedicated connection). Thebusiness logic engine 132 determines whether the content data requestedby the client 102 a belongs to a valid customer of the system 100 a or100 b, whether the rules of the customer allow the content data to beserviced to the client 102 a, whether the rules of the content deliverysystem 100 a or 100 b allow the content data to be serviced to theclient 102 a, and so on. The business logic engine 132 sends anauthentication response to the request service engine 122. Theauthentication response indicates to the request service engine 122whether the request received from the client 102 a can be serviced(e.g., “yes” or “no”). In some examples, in addition to the indicationof whether the request can be serviced, the authentication response caninclude metadata such as but not limited to, server ID, subscriber ID,cache key to use, and so on.

Whereas the caching agent 120 does not store a copy of the content datain the cache storage 126, the business logic agent (e.g., thefill-buffer proxy engine 134) receives from the caching agent 120 acontent request indicating that the caching agent 120 is requestingcontent data for the client 102, at 352. In some examples, the contentrequest includes an address of the content data and an address of anupstream node from which the content data is to be received/retrieved.

At 354, the fill-buffer proxy engine 134 fills the content data in thecache storage 136. The content data is received from the upstream node,the address of which is in the content request. For example, responsiveto receiving the content request, the fill-buffer proxy engine 134connects to the upstream node identified in the content request andreceives the content data from the upstream node.

At 356, the fill-buffer proxy engine 134 provides the cached contentdata to the caching agent 120 (e.g., the caching engine 124). Thefill-buffer proxy engine 134 begins to fill the content data in thecache storage 136 (e.g., at 354) and relays the content data to thecaching agent 120 via the port or connection of the network 140 a or 140b. Accordingly, the cache storage 126 is being filled with the contentdata from the upstream node via the fill-buffer proxy engine 134, whichis acting as a proxy or relay between the upstream node and the edgenode 110.

While the cache storage 126 is being filled with the content data, thebusiness logic agent 134 (e.g., the fill-buffer proxy engine 134)maintains the cached content data in the cache storage 136 in responseto receiving additional content requests from the caching agent 120. Theadditional content requests indicate that the caching agent 120 isrequesting the same content data for additional clients.

In one example implementation, at 358, the business logic agent 134(e.g., the fill-buffer proxy engine 134) determines whether anadditional content request has been received from the caching agent 120.In some examples, the determination at 358 is based on a predeterminedtime period since the last content request for the same content data isreceived by the fill-buffer proxy engine 134. In other words, the cachedcontent data is maintained in the cache storage 136 in response todetermining that two consecutive ones of the additional content requestsare received within a predetermined time interval or the content requestreceived at 352 and an additional content request are received within apredetermined time interval. Examples of the predetermined time intervalinclude but are not limited to, 1 s, 5 s, 10 s, 30 s, 1 m, 2 m, 10 m,and so on.

Implementing the predetermined time interval addresses a “race problem”which involves a situation where the fill using the cache storage 136 iscompleted, and the caching engine 124 has not released the cache lock(e.g., the fill using the cache storage 126 has not completed).

That is, the fill using the cache storages 126 and 136 may end atdifferent times. In this situation, if the fill-buffer proxy engine 134releases the content data from the cache storage 136 immediately afterthe fill being completed, the fill-buffer proxy engine 134 may need toestablish a new connection to the upstream node and re-fill the cachestorage 136. Accordingly, the predetermined time interval corresponds toa linger time before the fill-buffer proxy engine 134 releases thecached copy, thus providing efficient use of the cached copy to serve asmany additional content requests as possible.

In response to determining that no additional content request for thesame content data is received (358: NO), the business logic agent 134(e.g., the fill-buffer proxy engine 134) can release the cached contentdata from the cache storage 136, at 360. That is, responsive todetermining that no new additional content requests have been receivedfor the predetermined time interval since a most recent one of theadditional content requests has been received, the business logic agent134 (e.g., the fill-buffer proxy engine 134) can release the cachedcontent data from the cache storage 136, at 360.

On the other hand, in response to determining that an additional contentrequest (corresponding to a request from the client 102 b) for the samecontent data is received (358: YES), the business logic agent 134 (e.g.,the fill-buffer proxy engine 134) maintains the cached content data, a362. That is, responsive to determining that a new additional contentrequest has been received within the predetermined time interval since amost recent one of the additional content requests has been received,the business logic agent 134 (e.g., the fill-buffer proxy engine 134)maintains the cached content data, a 362.

As described, for each additional content request, a connection (port ordedicated connection) corresponding to that particular request isestablished in the network 140 a or 140 b. In some examples, so as longas a new port or dedicated connection is opened in the network 140 a or140 b within the predetermined time interval since a last port ordedicated connection has been open, the fill-buffer proxy engine 134maintains the cached content data in the cache storage 136. Each newport or dedicated connection can be used for communicating data betweenthe caching agent 120 and the business logic agent 130 forauthentication in the manner described. The block 362 is performed inresponse to successful authentication. The additional content requestcan be received using the same port or dedicated connection establishedfor authentication.

The cached content data is provided by the fill-buffer proxy engine 134to the client 102 b at 364, while the cache storage 126 is being filledwith the content data. Similar to the content request received at 352,the additional content request includes an address of the content dataand an address of an upstream node from which the content data is to bereceived/retrieved. In some examples, the additional content request mayalso include a range header for distinguishing a slice of the contentdata in the manner described. Using the address of the content data, theaddress of the upstream node, and the range header (if any), thefill-buffer proxy engine 134 can identify the content data (or a portionthereof) and the node to retrieve the content data using the additionalcontent request.

In the example in which the additional content request identifies thesame upstream node, the fill-buffer proxy engine 134 has alreadyestablished a connection with the upstream node for processing the firstcontent request with respect to the client 102 a. The same connectionbetween the fill-buffer proxy engine 134 and the upstream node can beleveraged to service the same content data to the additional requestfrom the client 102 b. In the example in which the content requestidentifies a different upstream node, the fill-buffer proxy engine 134is configured to establish a connection with the different upstream nodefor receiving the content data.

In some embodiments, the block 364 includes connecting directly to theclient 102 b (e.g., via a direct network connection without relayingthrough the caching agent 120) and sending, to the client 102 b, thecontent data from the cache storage 136 while the cache lock is enabledat the cache storage 126. In alternative embodiments, the block 364includes sending the content data to the caching agent 120 via the portor connection established in the network 140 a or 140 b (as dedicated tothe client 102 b) for authentication, so that the caching agent 120 canrelay the content data to the client 102 b. In some examples, thededicated tunnel can be established from the fill-buffer proxy engine134 to the client 102 b via the port or connection established forauthentication and through the caching agent 120. The content data canbe transferred from the fill-buffer proxy engine 134 to the client 102 bvia the dedicated tunnel.

The method 300 b returns to block 358 for determining whether anyadditional content requests have been received.

In some cases, another “race problem” may occur. This “race problem”concerns a situation in which the fill using the cache storage 126 iscomplete (e.g., servicing the request by the client 102 a is completedby the caching agent 120), and the fill-buffering proxy engine 134 hasnot released the content data (e.g., the fill using the cache storage136 for servicing the request by another one of the clients 102 a-102 nhas not been completed). At this point, the cached content data storedat the cache storage 126 may be expired, may be invalidated, or may bein need of update/refresh for another suitable reason. In this case, amechanism is needed to assure that the caching agent 120 does not send anew content request to the fill-buffer proxy engine 134 for a fill thatthe fill-buffer proxy engine 134 is already performing. That is, themechanism is needed to prevent the fill-buffer proxy engine 134 to usean expired/invalidated copy of the content data for addressing the newcontent request.

As such, the content request received by the fill-buffer proxy engine134 includes a fill ID in some embodiments. The fill ID indicates that anew, fresh, unexpired, and valid copy of the content data is requestedby the caching agent 120 as the caching agent 120 currently has anoutdated copy. The fill ID can be a header in the content request. Afirst fill ID indicates that any copy, even if not a new copy, of thecontent data can be used. A second fill ID indicates that a new copy ofthe content data needs to be used, so that the fill-buffer proxy engine134 needs to retrieve the new copy of the content data from the upstreamnode using a new connection thereto. As such, the fill-buffer proxyengine 134 can determine that a new copy of the content data is neededbased on the fill ID. The fill-buffer proxy engine 134 fills the newcopy of the content data in the cache storage 136 in response todetermining that the new copy is needed.

In some examples, the fill-buffer proxy engine 134, by using the fill IDheader, does not need to implement any expiration logic of its own. Thisis true regardless of whether the fill-buffer proxy engine 134 isimplementing very short-term caching. Even if the fill-buffer proxyengine 134 were doing long-term caching, using the fill ID header todifferentiate content data requested means that once content data hasexpired, the fill-buffer proxy engine 134 requests a new copy of thecontent data (based on receiving a new fill ID), and the old (prior thefill ID) content data will then age out of the cache storage 136 giventhat the old content data is no longer being requested. Therefore, thefill ID header does not only solve a race, but also simplifies theprocessing in the fill-buffer proxy engine 134 by obviating the need forthe fill-buffer proxy engine 134 to understand HTTP expiration.

In some embodiments, a conditional request header (or conditionalheader) can be used as a part of a key calculation. For example,responsive to the content copy being expired while being stored in thecache storage 126, the caching engine 124 makes a conditional requestupstream for the content data. The conditional request carries an HTTPheader indicating that the content data has expired, and that a new copyof the content data is needed if the content data has been changed sincethe expiration, and that if the content data has not been changed, aresponse indicative of the same is requested (so I can use the same).Traditionally, as the cache lock is disabled and the caching agent 120is receiving multiple copies of the same content data, the caching agent120 sends multiple conditional requests for the content data.

In present disclosure, the conditional request headers are included inthe key logic for the fill-buffer proxy engine 134 for differentiatingupstream fills, to ensure that the caching agent 120 can receive theresponse that the content data has not been changed from the fill-bufferproxy engine 134. In response to the fill-buffer proxy engine 134determining that a full version of the content data is already beingfilled and that a conditional request is received as part of the contentrequest, the fill-buffer proxy engine 134 responds to the contentrequest in the manner described using the copy of the content data thatis currently stored in the cache storage 136. In response to thefill-buffer proxy engine 134 determining that the content data is notbeing filled, and that a conditional request is received as part of thecontent request, then the fill-buffer proxy engine 134 uses theconditional request as a key (or as part of the key calculation).

The key is used for the fill-buffer proxy engine 134 to differentiatemultiple content requests for multiple resources. For example, thefill-buffer proxy engine 134 calculates a key for each content requestreceived. The key can be determined by running various parameters (suchas but not limited to, the target upstream node from which the contentdata is to be retrieved, the host/client that is requested the contentdata, the address (e.g., URL) of the content data, the fill ID, aninitial header, and so on) through a suitable key-generation function.

Accordingly, the content request includes a conditional request header.The fill-buffer proxy engine 134 determines a key corresponding to thecontent request based on the conditional request header.

The embodiments described herein have been described with reference todrawings. The drawings illustrate certain details of specificembodiments that implement the systems, methods and programs describedherein. However, describing the embodiments with drawings should not beconstrued as imposing on the disclosure any limitations that may bepresent in the drawings.

It should be understood that no claim element herein is to be construedunder the provisions of 35 U.S.C. § 112(f), unless the element isexpressly recited using the phrase “means for.”

As used herein, the term “circuit” may include hardware structured toexecute the functions described herein. In some embodiments, eachrespective “circuit” may include machine-readable media for configuringthe hardware to execute the functions described herein. The circuit maybe embodied as one or more circuitry components including, but notlimited to, processing circuitry, network interfaces, peripheraldevices, input devices, output devices, sensors, etc. In someembodiments, a circuit may take the form of one or more analog circuits,electronic circuits (e.g., integrated circuits (IC), discrete circuits,system on a chip (SOCs) circuits, etc.), telecommunication circuits,hybrid circuits, and any other type of “circuit.” In this regard, the“circuit” may include any type of component for accomplishing orfacilitating achievement of the operations described herein. Forexample, a circuit as described herein may include one or moretransistors, logic gates (e.g., NAND, AND, NOR, OR, XOR, NOT, XNOR,etc.), resistors, multiplexers, registers, capacitors, inductors,diodes, wiring, and so on).

The “circuit” may also include one or more processors communicativelycoupled to one or more memory or memory devices. In this regard, the oneor more processors may execute instructions stored in the memory or mayexecute instructions otherwise accessible to the one or more processors.In some embodiments, the one or more processors may be embodied invarious ways. The one or more processors may be constructed in a mannersufficient to perform at least the operations described herein. In someembodiments, the one or more processors may be shared by multiplecircuits (e.g., circuit A and circuit B may comprise or otherwise sharethe same processor which, in some example embodiments, may executeinstructions stored, or otherwise accessed, via different areas ofmemory). Alternatively or additionally, the one or more processors maybe structured to perform or otherwise execute certain operationsindependent of one or more co-processors. In other example embodiments,two or more processors may be coupled via a bus to enable independent,parallel, pipelined, or multi-threaded instruction execution. Eachprocessor may be implemented as one or more general-purpose processors,ASICs, FPGAs, DSPs, or other suitable electronic data processingcomponents structured to execute instructions provided by memory. Theone or more processors may take the form of a single core processor,multi-core processor (e.g., a dual core processor, triple coreprocessor, quad core processor, etc.), microprocessor, etc. In someembodiments, the one or more processors may be external to theapparatus, for example the one or more processors may be a remoteprocessor (e.g., a cloud based processor). Alternatively oradditionally, the one or more processors may be internal and/or local tothe apparatus. In this regard, a given circuit or components thereof maybe disposed locally (e.g., as part of a local server, a local computingsystem, etc.) or remotely (e.g., as part of a remote server such as acloud based server). To that end, a “circuit” as described herein mayinclude components that are distributed across one or more locations.

An exemplary system for implementing the overall system or portions ofthe embodiments might include a general purpose computing computers inthe form of computers, including a processing unit, a system memory, anda system bus that couples various system components including the systemmemory to the processing unit. Each memory device may includenon-transient volatile storage media, non-volatile storage media,non-transitory storage media (e.g., one or more volatile and/ornon-volatile memories), etc. In some embodiments, the non-volatile mediamay take the form of ROM, flash memory (e.g., flash memory such as NAND,3D NAND, NOR, 3D NOR, etc.), EEPROM, MRAM, magnetic storage, hard discs,optical discs, etc. In other embodiments, the volatile storage media maytake the form of RAM, TRAM, ZRAM, etc. Combinations of the above arealso included within the scope of machine-readable media. In thisregard, machine-executable instructions comprise, for example,instructions and data which cause a general purpose computer, specialpurpose computer, or special purpose processing machines to perform acertain function or group of functions. Each respective memory devicemay be operable to maintain or otherwise store information relating tothe operations performed by one or more associated circuits, includingprocessor instructions and related data (e.g., database components,object code components, script components, etc.), in accordance with theexample embodiments described herein.

It should also be noted that the term “input devices,” as describedherein, may include any type of input device including, but not limitedto, a keyboard, a keypad, a mouse, joystick or other input devicesperforming a similar function. Comparatively, the term “output device,”as described herein, may include any type of output device including,but not limited to, a computer monitor, printer, facsimile machine, orother output devices performing a similar function.

It should be noted that although the diagrams herein may show a specificorder and composition of method steps, it is understood that the orderof these steps may differ from what is depicted. For example, two ormore steps may be performed concurrently or with partial concurrence.Also, some method steps that are performed as discrete steps may becombined, steps being performed as a combined step may be separated intodiscrete steps, the sequence of certain processes may be reversed orotherwise varied, and the nature or number of discrete processes may bealtered or varied. The order or sequence of any element or apparatus maybe varied or substituted according to alternative embodiments.Accordingly, all such modifications are intended to be included withinthe scope of the present disclosure as defined in the appended claims.Such variations will depend on the machine-readable media and hardwaresystems chosen and on designer choice. It is understood that all suchvariations are within the scope of the disclosure. Likewise, softwareand web implementations of the present disclosure could be accomplishedwith standard programming techniques with rule based logic and otherlogic to accomplish the various database searching steps, correlationsteps, comparison steps and decision steps.

The foregoing description of embodiments has been presented for purposesof illustration and description. It is not intended to be exhaustive orto limit the disclosure to the precise form disclosed, and modificationsand variations are possible in light of the above teachings or may beacquired from this disclosure. The embodiments were chosen and describedin order to explain the principals of the disclosure and its practicalapplication to enable one skilled in the art to utilize the variousembodiments and with various modifications as are suited to theparticular use contemplated. Other substitutions, modifications, changesand omissions may be made in the design, operating conditions andembodiment of the embodiments without departing from the scope of thepresent disclosure as expressed in the appended claims.

What is claimed is:
 1. A method for managing caching for a contentdelivery system, the method comprising: receiving, by a business logicagent from a caching agent, a content request indicating that thecaching agent is requesting content data for a client, wherein thebusiness logic agent and the caching agent are operatively coupled via anetwork; filling, by the business logic agent, the content data in afirst cache storage of the business logic agent, wherein the contentdata is received from an upstream node of the content delivery system;providing, by the business logic agent, the cached content data to thecaching agent; while a second cache storage of the caching agent isbeing filled with the content data, maintaining, by the business logicagent, the cached content data in response to receiving additionalcontent requests from the caching agent, wherein the additional contentrequests indicate that the caching agent is requesting the same contentdata for additional clients; receiving, by the business logic agent fromthe caching agent, an authentication request before the content requestis received; authenticating that the content data is provided by acustomer of the content delivery system; and sending, by the businesslogic agent to the caching agent, an authentication response in responseto authenticating that the content data is provided by the customer,wherein the authentication response comprises an authorization.
 2. Themethod of claim 1, wherein the content delivery system is a contentdelivery network (CDN); the business logic agent and the caching agentare on a same edge node of the CDN; the upstream node is an originserver that stores the content data or a node between the edge node andthe origin node in the CDN that stores the content data; and the networkis a local host connection.
 3. The method of claim 2, wherein the localhost connection is over at least one of a hypertext transfer protocol(HTTP) connection and a HTTP/2 connection.
 4. The method of claim 1,wherein the content delivery system is a content delivery network (CDN);the caching agent is on an edge node of the CDN; the business logicagent is on an intermediate node of the CDN, wherein the intermediatenode is different from the edge node; and the upstream node is an originserver that stores the content data or a node between the edge node andthe origin node in the CDN that stores the content data.
 5. The methodof claim 1, wherein the caching agent comprises a hypertext transferprotocol (HTTP) service engine configured to service HTTP requestsreceived from clients; and the caching agent comprises a caching engine,wherein the caching engine comprises the second cache storage.
 6. Themethod of claim 1, wherein the content request identifies the upstreamnode; and the business logic agent receives the content data from theupstream node based on the content request.
 7. The method of claim 1,wherein the content request is addressed to the upstream node.
 8. Themethod of claim 1, wherein the content request comprises an address ofthe content data and a range of the content data; the address of thecontent data comprises a Uniform Resource Locator (URL); and thebusiness logic agent determines whether any of the additional contentrequests is requesting the same content data as requested by the contentrequest based on the address and the range.
 9. The method of claim 1,wherein the content request comprises a fill identification (ID), andthe method further comprises: determining, by the business logic agent,that a new copy of the content data is needed based on the fill ID; andfilling, by the business logic agent, the new copy of the content datain the first cache storage in response to determining that the new copyis needed.
 10. The method of claim 1, wherein the content requestcomprises a conditional request header, and the method furthercomprises: determining, by the business logic agent, a key correspondingto the content request based on the conditional request header.
 11. Themethod of claim 1, wherein while the second cache storage of the cachingagent is being filled with the content data, maintaining the cachedcontent data comprises maintaining the cached content data in responseto determining that two consecutive ones of the additional contentrequests are received within a predetermined time interval.
 12. Themethod of claim 1, further comprising determining that no new additionalcontent requests have been received for a predetermined time intervalsince a most recent one of the additional content requests has beenreceived.
 13. A method for managing caching for a content deliverysystem, the method comprising: receiving, by a caching agent from aclient, a request for content data; determining, by the caching agent,that the caching agent needs to fill the content data; sending, by thecaching agent to a business logic agent, a content request indicatingthat the caching agent is requesting the content data for the client,wherein the business logic agent and the caching agent are operativelycoupled via a network, and the business logic agent fills the contentdata in a first cache storage of the business logic agent responsive tothe content request; filling, by the caching agent, the content datafrom an upstream node in a second cache storage of the caching agent;receiving, by the caching agent, additional content requests fromadditional clients, wherein the additional content requests indicatethat the same content data is requested by additional clients; while thesecond cache storage is being filled with the content data, the cachedcontent data is provided by the business logic agent to the additionalclients; sending, by the caching agent to the business logic agent, anauthentication request before the content request is sent; andreceiving, by the caching agent from the business logic agent to anauthentication response, wherein the authentication response comprisesan authorization.
 14. The method of claim 13, wherein the contentdelivery system is a content delivery network (CDN); the business logicagent and the caching agent are on a same edge node of the CDN; theupstream node is an origin server that stores the content data or a nodebetween the edge node and the origin node in the CDN that stores thecontent data; and the network is a local host connection.
 15. The methodof claim 14, wherein the local host connection is over at least one of ahypertext transfer protocol (HTTP) connection and a HTTP/2 connection.16. The method of claim 13, wherein the content delivery system is acontent delivery network (CDN); the caching agent is on an edge node ofthe CDN; the business logic agent is on an intermediate node of the CDN,wherein the intermediate node is different from the edge node; and theupstream node is an origin server that stores the content data or a nodebetween the edge node and the origin node in the CDN that stores thecontent data.
 17. The method of claim 13, wherein the caching agentcomprises a hypertext transfer protocol (HTTP) service engine configuredto service HTTP requests received from clients; and the caching agentcomprises a caching engine, wherein the caching engine comprises thesecond cache storage.
 18. The method of claim 13, further comprisingselecting, by the caching agent, the upstream node, wherein the contentrequest identifies the upstream node.
 19. The method of claim 13,wherein the content request is addressed to the upstream node.
 20. Themethod of claim 13, wherein the content request comprises an address ofthe content data and a range of the content data; the address of thecontent data comprises a Uniform Resource Locator (URL); and thebusiness logic agent determines whether any of the additional contentrequests is requesting the same content data as requested by the contentrequest based on the address and the range.
 21. The method of claim 13,wherein the content request comprises a fill identification (ID), andthe fill ID indicates whether a new copy of the content data is neededto be filled in the first cache storage of the business logic agent. 22.The method of claim 13, wherein the content request comprises aconditional request header, wherein a key corresponding to the contentrequest is determined by the business logic agent based on theconditional request header.
 23. A node of a content delivery system,comprising: a network device providing a network; at least one processorimplementing a business logic agent and a caching agent, the businesslogic agent and the caching agent are operatively coupled via thenetwork; and at least one memory implementing a first cache storage ofthe business logic agent and a second cache storage of the cachingagent, wherein the at least one processor: sends, using the cachingagent to the business logic agent, a content request indicating that thecaching agent is requesting content data for a client; receives, usingthe business logic agent, the content data from an upstream node of thecontent delivery system; fills, using the business logic agent, thecontent data in the first cache storage; provides, using the businesslogic agent, the cached content data to the client; while the secondcache storage is being filled with the content data, maintains, usingthe business logic agent, the cached content data in response to thebusiness logic agent receiving additional content requests from thecaching agent, wherein the additional content requests indicate that thecaching agent is requesting the same content data for additionalclients; receives, by the business logic agent from the caching agent,an authentication request before the content request is received;authenticates that the content data is provided by a customer of thecontent delivery system; and sends, by the business logic agent to thecaching agent, an authentication response in response to authenticatingthat the content data is provided by the customer, wherein theauthentication response comprises an authorization.
 24. A non-transitorycomputer-readable medium comprising computer-readable instructions suchthat, when executed, cause a processor of a node of a content deliverysystem to: send, using a caching agent to a business logic agent, acontent request indicating that the caching agent is requesting contentdata for a client; receive, using the business logic agent, the contentdata from an upstream node of the content delivery system; fill, usingthe business logic agent, the content data in the first cache storage;provide, using the business logic agent, the cached content data to theclient; while the second cache storage is being filled with the contentdata, maintain, using the business logic agent, the cached content datain response to the business logic agent receiving additional contentrequests from the caching agent, wherein the additional content requestsindicate that the caching agent is requesting the same content data foradditional clients; receives, by the business logic agent from thecaching agent, an authentication request before the content request isreceived; authenticates that the content data is provided by a customerof the content delivery system; and sends, by the business logic agentto the caching agent, an authentication response in response toauthenticating that the content data is provided by the customer,wherein the authentication response comprises an authorization.